A report posted on the website of The Wellness Company has been generating considerable social media attention for a new cancer treatment with announcements such as these: 84% of patients experienced “clinical benefit” and nearly a third showed “no current evidence of disease.” Before you share this with desperate friends and relatives, it is important to look at how these numbers were derived, what they mean, and what they do not.
What is the Wellness Company
The Wellness Company (TWC) sells supplements and prescription medications via a network of tele-health providers. Peter McCullough, MD, MPH, is the chief scientific officer of TWC. You may recall Dr. McCullough as a hydroxychloroquine evangelist, turned ivermectin evangelist, turned general COVID contrarian. Among TWC’s prescription medications is a combination ivermectin 25 mg/mebendazole 250 mg capsule. Both drugs are FDA-approved as anti-parasitics.
All of the authors of the paper are affiliated with and/or salaried by the Wellness Company. This is disclosed openly. It is not unusual for authors to have such conflicts of interest, especially in privately funded studies. It is useful context in interpreting results.
The Study
Subjects in this report were patients with a diagnosis of cancer who were prescribed TWC’s combination of ivermectin and mebendazole. The manuscript states that these were confirmed cancer cases, but because this report is based on self-reported data, it is not clear what is meant by “confirmed.” These medications were prescribed for off-label use by licensed providers affiliated with TWC. Off-label prescription of medications is considered within the scope of medical practice and not prohibited by the Food and Drug Administration (FDA). However, research involving the off-label use of medications is often regulated.
Patients were invited to participate, and participation was optional. Volunteers were sent electronic surveys at the time the prescription was filled and again 6 months later.
No TWC doctors examined these patients for the study. No scans were ordered or reviewed. No blood tests were ordered or reviewed. The oncologists caring for these patients were not contacted to confirm the cancer diagnosis or verify the status of the cancer. The cancer diagnosis, the disease status, the reported outcomes all came from patients answering questionnaires on electronic devices.
Results
197 patients completed the baseline survey, and 122 at 6 months. The most common cancers were prostate (27.9%), breast (18.3%), and lung (8.6%), with a wide variety of other malignancies. Most patients had already undergone conventional therapy (such as surgery, chemotherapy, radiation). There were only 2 choices for the status of their cancer at baseline: Actively progressing (37.1%) or not currently spreading (62.9%).
At 6 months, there were 122 responses (61.9% of the original patients). Data included medication dose and adherence, concomitant treatments, and various measures of cancer control. 1/3 of the patients who responded at 6 months were no longer taking the medication. A large number of patients were using other forms of therapy for their cancer, some conventional (chemotherapy, radiation), and others such as supplements, diets, etc. Compared to the baseline survey, a different scheme for classifying disease was used at 6 months: No current evidence of disease (NED) (32.8%), regressed (15.6%), stable disease (36.1%), spread or progressed (15.6%). Some of these results were combined into outcomes such as: NED+Regression (48.4%), as well as Clinical Benefit Ratio (CBR) (the sum of NED+Regression+Stable) (84.4%)
Problems with the data and analysis
- No control group. This was a highly diverse group of cancer patients, most of whom had undergone, or were undergoing, cancer treatment at the time of enrollment. Many continued to receive treatment during the 6 months of the study. Over 60% reported their cancer “not currently spreading” at baseline. How would this group, or a similar group, behave without the ivermectin/mebendazole? Without a control group, it is impossible to know if treatment with ivermectin/mebendazole influenced the status of the cancers in any meaningful way.
- Self-reported outcomes. Oncology studies typically assess cancer status using strictly defined, pre-specified criteria based on physician assessment, imaging studies, and other cancer biomarkers. TWC authors assessed cancer status based solely on patient-reported data. We have no idea if or when each patient had a systematic search for cancer or when they last saw a qualified oncologist. The TWC study uses the same terminology as high-quality clinical trials, such as: NED, regression, and Clinical Benefit Ratio, but without the standardization and rigor. The “Clinical Benefit Ratio” deserves special attention because it is a headline statistic in TWC’s dissemination of the study results. In legitimate cancer research, CBR is used in studies of aggressive metastatic cancers where most patients would be expected to deteriorate rapidly without treatment. It requires repeated imaging studies at specified intervals to confirm. For the TWC study, 62.9% of patients reported “not currently spreading” at baseline—a very different patient population than is usual for a CBR endpoint.
- The missing 38%. The fact that over 1/3 of the patients did not participate in the 6-month survey is a serious red flag. Patients who leave a study tend to differ from those who remain, a phenomenon known as attrition bias. Patients who are doing poorly are more likely to leave a study, and those doing well are more likely to remain. Among the TWC patients who did not respond at 6 months: How many died? Dead patients do not reply to questionnaires. How many were infirm due to progression of their cancer? How many dropped out due to real or perceived progression of their cancer? How many had intolerable side effects?
Each of these three flaws individually is an enormous weakness. Collectively, they render this study incapable of supporting any conclusions about the efficacy of ivermectin/mebendazole in cancer treatment. The treatment could be helpful, harmful, or indifferent.
What is human research?
Beyond the sloppiness of this study, there are other serious concerns.
Despite the fact that this looks like human research and smells like human research, and is reported like human research, TWC authors want you to know that their project is certainly not human research. Here is the statement from their article:
Institutional Review Board (IRB) Statement: This project was conducted as a retrospective analysis of a prospective clinical program evaluation. Consistent with established frameworks for quality improvement and internal assessment of clinical services, the project utilized voluntary, patient-reported data. As such, it did not meet the definition of human subjects research and did not require Institutional Review Board (IRB) review.
Why would they make such a declaration? There are certain rules and expectations about how human research is conducted. If this is indeed “not human research”, those rules would not apply.
Definitions and Regulation of Human Research
There are a few entities within the Department of Health and Human Services (HHS) that are responsible for the oversight of research. The Office for Human Research Protections (OHRP) regulates HHS-funded human subjects research, including strict rules for IRBs and rules for investigators performing human research. The policy for protection of human subjects is codified in 45 CFR 46, also known as the Common Rule.
Here are the relevant definitions:
Human subject means a living individual about whom an investigator (whether professional or student) conducting research:
(i) Obtains information or biospecimens through intervention or interaction with the individual, and uses, studies, or analyzes the information [emphasis added] or biospecimens; or
(ii) Obtains, uses, studies, analyzes, or generates identifiable private information [emphasis added] or identifiable biospecimens.
Research means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.
What is an Institutional Review Board?
TWC investigators asserted that their work did not require Institutional Review Board (IRB) review because it was not human research.
An Institutional Review Board (IRB) is a committee that reviews research involving human participants. Its job is to protect the people in the study. Before a research study begins, the IRB reviews the plan in detail, how participants will be recruited, what risks they face, what they will be told, and how their privacy will be protected. Most importantly, the IRB oversees the informed consent process, ensuring that participants genuinely understand they are in a research study and agree to participate with full informed consent.
- They would need to be informed of risks, including possible loss of confidentiality.
- They would need to be informed that their participation is voluntary, that there is no penalty for declining participation, and that they can withdraw consent at any time.
- They would need to be informed whether or not their data will still be used if they decide to withdraw from the study.
- The investigators would need to declare whether any personal health information would be stored and whether the study records could be linked back to individual patients.
Because TWC determined that their study did not qualify, we do not know to what extent the patients were afforded these protections.
What is quality improvement?
TWC authors classify their project as quality improvement instead of human research. Quality improvement (QI) is a legitimate and important category of investigation. QI activities are usually distinct from research and typically do not require IRB oversight.
Quality improvement is not defined as rigorously as research. Here is what CMS has to say:
Quality improvement is the framework used to systematically improve care. Quality improvement seeks to standardize processes and structure to reduce variation, achieve predictable results, and improve outcomes for patients, healthcare systems, and organizations. Structure includes things like technology, culture, leadership, and physical capital; process includes knowledge capital (e.g., standard operating procedures) or human capital (e.g., education and training).
In other words, quality improvement is an exercise within an institution to improve the delivery of healthcare. Quality improvement is meant to identify and solve problems within an organization or healthcare system. In contrast, research is meant to contribute to generalizable knowledge.
Examples of things that would be quality improvement projects:
- Studying how the location of hand-sanitizer dispensers influences compliance with institutional hygiene policy.
- Tracking visits to figure out how to decrease patient wait times.
- Studying missed or cancelled appointments to develop strategies to improve patient follow-up.
For a telemedicine company like TWC, quality improvement projects might be:
- Tracking how quickly providers respond to patient messages.
- Comparing strategies to ensure prescriptions are filled accurately and promptly.
- Questionnaires to determine how well patients understand how to take their medications?
What is the nature of this study?
A project does not become “not research” because the authors declare it to be so. Rules and definitions determine whether or not a project is human research, and sometimes the parameters are ambiguous or borderline. If the nature of the study is ambiguous, investigators can go to an IRB in advance and get an expert opinion. IRBs issues these types of advisory opinions regularly.
In my opinion, the identification of this as a quality improvement project is incorrect and disingenuous. The questionnaire did not ask questions relevant to a quality improvement project. The questions were relevant to the efficacy and safety of their ivermectin/ mebendazole combination treatment.
In practice, the authors don’t seem to believe their own opinion. Let’s dissect this paragraph from the paper’s Introduction.
Despite compelling preclinical data and documented safe use in cancer patients [4], robust clinical evidence evaluating the ivermectin–mebendazole combination in oncology remains limited. The present prospective clinical program evaluation was therefore conducted to assess real-world self-reported cancer outcomes, medication adherence, tolerability, and patient experience among individuals prescribed a compounded ivermectin–mebendazole formulation through a telemedicine platform.
They make the case for ivermectin and mebendazole as a cancer treatment based on preclinical and early clinical data. They frame a gap in the current knowledge about the two drugs. The next sentence is an explicit explanation that the “prospective clinical program” was to assess cancer outcomes, etc.
In the “conclusions” section of the abstract, they say:
These findings provide a compelling clinical signal that these well-tolerated, repurposed agents may offer therapeutic benefit … these findings should be interpreted as hypothesis-generating.
Later in the paper, they use these results to advocate the need for urgent randomized clinical trials.
“Clinical signals” that are “hypothesis generating” and augment an appeal for further research are pretty definitive claims of generalizable knowledge, a key component of the definition of research.
Recall the carefully crafted wording on the preprint manuscript declaring that their study: “…did not meet the definition of human subjects research and did not require Institutional Review Board (IRB) review.”
And suddenly… it is research!
In an Instagram video, Nicolas Hulscher, MPH, hypes the “absolutely shocking” results of the study. At the end of the video are these words: “…this is all very, very promising research.” Nicolas Hulscher is not a random Instagram influencer. He is the first author of the study. An author who had preemptively asserted that this project was not research is now declaring it as “very, very promising research.”
Is this study under the jurisdiction of the FDA?
The FDA operates in a different regulatory lane from the Common Rule. Many of the practices are harmonized, but regulatory requirements differ in many ways.
Because this research is privately funded, TWC has a potential loophole from IRB oversight. The Common Rule explicitly applies to government-funded research. Unlike the Common Rule, FDA regulations apply regardless of the funding source.
Whether or not this project is under their jurisdiction would ultimately be determined by the FDA. Things are especially complicated for this investigation because it involves a compounded medication, which involves its own set of regulations.
The FDA often requires the sponsor to file an Investigative New Drug application (IND) for certain investigations. An IND is required for studies of unapproved new drugs seeking FDA approval, and often for approved drugs that are being used outside the constraints of the current FDA approval (such as dose, route, or indication). Filing an IND application is a major effort.
There is a list of exceptions for an IND, and if ALL the exceptions are met, the research is exempt. Among the exemptions are these.
The clinical investigation does not involve a route of administration, dose, patient population, or other factor that significantly increases the risk (or decreases the acceptability of the risk) associated with the use of the drug.
Both ivermectin and mebendazole are FDA-approved for the treatment of certain parasitic infections.
In the FDA-approved form, Ivermectin comes in 3 mg tablets, and dosing is 200 µg/kg for 1 day. A 75 kg (165 lb) individual would receive a single 15 mg dose.
Mebendazole comes in 100 mg tablets, and the dose ranges from 1 pill for one day to one pill twice a day for 3 days.
The TWC formulation was 25 mg ivermectin and 250 mg mebendazole. Cancer patients were prescribed 1 or 2 pills per day for up to six months. This checks many of the boxes usually necessitating an IND: off-label indication (cancer vs parasitic infection); doses up to several multiples of the labelled doses; durations that exceeded the label 20-fold or more.
Since this is a compounded medication, however, different rules apply. Compounded medications are not evaluated or approved by the FDA for any indication.
Center Watch is a company focused on the clinical trials industry. They provide a variety of services to clinical investigators, including guidance in fulfilling regulatory requirements for research. They have a page specifically about IND regulation of compounded drugs. Here are some relevant quotes. “Modifications to the lawfully marketed drug” refers to compounded drugs.
Research involving modifications to the lawfully marketed drug may also be IND-exempt, provided they do not introduce risks exceeding those associated with the product’s standard, labeled use. This exemption would typically apply to low-risk modifications to the lawfully marketed dosage form, such as minor variations to solid oral dosage forms. For example, changes to the color, scoring, or capsule size of the marketed dosage form for blinding purposes would generally be considered low risk, if they do not involve significant manufacturing or formulation changes.
While sponsors are primarily responsible for determining the need for an IND (21 CFR 312.2(a)), Institutional Review Boards (IRBs) play a critical role in ensuring study compliance with regulations, protecting the rights and welfare of participants, and assessing that the research does not introduce new risks (21 CFR 56.101). IRBs rely on sponsors and investigators to provide the documentation to support IND exemption claims.
Ultimately, the FDA has the final authority to determine the requirements for investigating the compounded drug in the research. Sponsors and investigators should proactively consult the FDA when studying compounded versions of lawfully marketed drugs to determine IND requirements.
Summary and opinion
In addition to being an investigator on numerous human research projects, I have many years of service on IRBs at two research institutions. I cannot make a definitive ruling on how an IRB or the FDA would regulate this study, but I can come to an informed opinion.
Declaring this work as a quality improvement project, in my opinion, was not credible. It insulated them from the motivation to seek review from an IRB. Had they sought such an opinion, it is likely that an IRB would have declared this a human research project. It is also likely that the IRB would have requested that the investigators present an IND, or an IND exemption from the FDA.
This is a shoddy research study presented as a quality improvement project. This dubious classification functioned as an excuse for lack of interface with relevant regulatory bodies.
There is validity to the assertion that ivermectin and mebendazole show anticancer activity in preclinical studies. This study provides no meaningful evidence for or against the efficacy of such treatments in humans with cancer.